Imaging device and imaging method

ABSTRACT

An imaging device includes: an imaging portion that captures an image of a portion of a living body to take in the image; a display portion that displays first and second display images with being superimposed on each other, the first display image being based on the taken-in image, the second display image including guidance regarding a way to place the portion of the living body in a prescribed position; a determination portion that determines whether the portion of the living body is placed in the prescribed position; and a control portion that, until it is determined that the portion of the living body is placed in the prescribed position, causes the imaging portion to newly capture and take in a new image of the portion of the living body and causes the display portion to display an image based on the new taken-in image.

TECHNICAL FIELD

The present invention relates to an imaging device and an imagingmethod.

BACKGROUND ART

Patent Document 1 shows an example of a personal authentication devicethat identifies an individual by using a finger vein pattern. In thepersonal authentication device described in Patent Document 1, in orderto provide guidance regarding the correct way of inserting a finger intothe imaging portion, an image showing the image capture state of thefinger and a contour guide that serves as a guide for aligning thecontour of the finger are displayed in an overlapped state on a monitor.In addition, information indicating how the user should correct thefinger state is displayed on this monitor using sentences and images.According to the personal authentication device described in PatentDocument 1, by the user correctly performing the registration workaccording to the guidance on the screen, the work performed by theadministrator at the time of registration will be reduced.

PRIOR ART DOCUMENTS Patent Documents

[Patent Document 1] Japanese Unexamined Patent Application, FirstPublication No. 2006-26427

SUMMARY OF THE INVENTION Problem to be Solved by the Invention

In the personal authentication device described in Patent Document 1,the correct way of inserting a finger into the imaging portion isindicated by sentences (text) and figures. For that reason, for example,if the user cannot comprehend the language of the sentences, theguidance may not be understood or the effect of the guidance may bereduced. Therefore, there is a problem in which an object such as afinger sometimes cannot be placed at the correct imaging position.

An example object of the present invention is to provide an imagingdevice and an imaging method capable of solving the above-mentionedproblems.

Means for Solving the Problem

One example aspect of the present invention is an imaging deviceincluding: an imaging portion that captures an image of a portion of aliving body to take in the image; a display portion that displays afirst display image and a second display image with being superimposedon each other, the first display image being based on the image taken inby the imaging portion, the second display image including guidanceregarding a way to place the portion of the living body in a prescribedposition; a determination portion that determines whether the portion ofthe living body is placed in the prescribed position; and a controlportion that, until the determination portion determines that theportion of the living body is placed in the prescribed position, causesthe imaging portion to newly capture and take in a new image of theportion of the living body and causes the display portion to display, asthe first display image, an image based on the new image taken in by theimaging portion.

Further, one example aspect of the present invention is an imagingmethod including: capturing an image of a portion of a living body totake in the image; displaying a first display image and a second displayimage with being superimposed on each other, the first display imagebeing based on the taken-in image, the second display image includingguidance regarding a way to place the portion of the living body in aprescribed position; determining whether the portion of the living bodyis placed in the prescribed position; and until it is determined thatthat the portion of the living body is placed in the prescribedposition, newly capturing and taking in a new image of the portion ofthe living body and displaying, as the first display image, an imagebased on the new taken-in image.

Effect of the Invention

According to each example aspect of the present invention, the user caneasily place a portion of a living body (for example, an object) in aprescribed position (for example, a correct imaging position).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic configuration diagram showing a configurationexample of an imaging device according to a first example embodiment ofthe present invention.

FIG. 2 is a perspective view schematically showing a configurationexample of an imaging device 10 according to a second example embodimentof the present invention.

FIG. 3 is a schematic view showing a configuration example of an imagingunit 1 according to the second example embodiment of the presentinvention.

FIG. 4 is a perspective view showing a configuration example of theimaging unit 1 shown in FIG. 3.

FIG. 5 is a schematic diagram for explaining an operation example of theimaging unit 1 shown in FIG. 3.

FIG. 6 is a schematic diagram for explaining an operation example of theimaging unit 1 shown in FIG. 3.

FIG. 7 is a plan view schematically showing a configuration example ofthe imaging unit 1 shown in FIG. 3.

FIG. 8A is a diagram showing an example of an image captured by theimaging unit 1 shown in FIG. 3.

FIG. 8B is a diagram showing an example of an image captured by theimaging unit 1 shown in FIG. 3.

FIG. 9 is a flowchart showing an operation example of the imaging device1 shown in FIG. 1.

FIG. 10 is a schematic diagram for explaining an operation example ofthe imaging device 1 shown in FIG. 1.

FIG. 11 is a flowchart showing another operation example of the imagingdevice 1 shown in FIG. 1.

EXAMPLE EMBODIMENT

Hereinbelow, example embodiments of the present invention will bedescribed with reference to the drawings. FIG. 1 is a schematicconfiguration diagram showing a configuration example of an imagingdevice according to a first example embodiment of the present invention.The imaging device 10 shown in FIG. 1 includes an imaging portion 11, adisplay portion 12, a control portion 13, and a determination portion 15that supplies an image determination result to the control portion 13. Afirst imaging portion 11 images an object to be imaged, such as aportion of a living body, placed at a predetermined imaging position.The display portion 12 displays a first display image representing animage of the portion of the living body captured by the imaging portionand a second display image representing an authenticatable area that canbe effectively authenticated from the captured image. The determinationportion 15 determines whether or not authentication by the first displayimage displayed on the display portion 12 is possible. The controlportion 13, by controlling the first imaging portion 11, the displayportion 12, and the determination portion 15, repeats the imaging of aportion of the living body by the first imaging portion 11 to displaythe first display image of the display portion 12 at the imaged positionuntil the determination portion 15 determines that determination by thefirst display image is possible.

According to the imaging device 1 shown in FIG. 1, since the user canrecognize whether or not the object is correctly placed at the imagingposition on the basis of a change in the display mode of the displayportion 12, the object can be easily placed in the correct imagingposition.

Next, the imaging device 1 according to a second example embodiment willbe described with reference to FIG. 2. FIG. 2 is a perspective viewschematically showing a configuration example of the imaging device 10shown in FIG. 1. In FIG. 2, those configurations corresponding to theconfigurations shown in FIG. 1 are given the same reference symbols. Inthe example shown in FIG. 2, the object to be imaged is a fingertipportion (the distal end of a person's finger). One purpose of theimaging device 10 is to capture a fingerprint on the ventral surface ofthe fingertip portion as a physical feature for biometricauthentication. However, the object to be imaged by the imaging device10 is not limited to a fingertip portion, and may be a palm or the like.That is, one purpose of the imaging device 10 may be to capture anotherphysical feature for biometric authentication such as a palm print.Further, the object of the imaging device 10 is not limited to a part ofthe body.

In the example shown in FIG. 2, the imaging device 10 includes a housing101 and a housing 102. The housing 101 is provided with a displayportion 12. The housing 102 is provided with two imaging units 1 a and 1b and the control portion 13. The imaging unit 1 a includes a firstimaging portion 11 a corresponding to the first imaging portion 11 shownin FIG. 1, and a light source portion and an optical system describedlater. The imaging unit 1 a captures by the first imaging portion 11 a afirst image representing a fingertip portion, which is an object placedat an imaging position 14 a. Similarly to the imaging unit 1 a, theimaging unit 1 b includes a first imaging portion 11 b corresponding tothe first imaging portion 11 shown in FIG. 1, and a light source portionand an optical system described later. The imaging unit 1 b captures bymeans of the first imaging portion 11 b a first image representing thefingertip portion, which is an object placed at an imaging position 14b. Each first imaging portion 11 may include one imaging portion(camera) to capture one type of first image, or may include two or moreimaging portions (cameras) to capture two types of images.

The display portion 12, which is for example a liquid crystal display oran organic electroluminescence display, displays an image in a displayarea 12 a and a display area 12 b under the control of the controlportion 13. Specifically, the display portion 12 displays in the displayarea 12 a a first display image representing the fingertip portionplaced at the imaging position 14 a, and a second display image showingguidance on how to place the fingertip portion at the imaging position14 a. The first display image is an image based on the first imageobtained by the first imaging portion 11 a imaging the fingertip portionplaced at the imaging position 14 a. Further, the display portion 12displays in the display area 12 b the first display image and the seconddisplay image showing guidance on how to place the fingertip portion atthe imaging position 14 b. The first display image is an image based onthe first image obtained by the first imaging portion 11 b imaging thefingertip portion placed at the imaging position 14 b.

The control portion 13 acquires the first image captured by the firstimaging portion 11 a, and determines whether or not the fingertipportion is correctly placed at the imaging position 14 a based on theacquired first image. The control portion 13 changes the display mode inthe display area 12 a of the display portion 12 depending on whether acase of the fingertip portion being correctly placed or a case of thefingertip portion being not correctly placed. Further, the controlportion 13 acquires the first image captured by the first imagingportion 11 b, and determines whether or not the fingertip portion iscorrectly placed at the imaging position 14 b based on the acquiredfirst image. The control portion 13 changes the display mode in thedisplay area 12 b of the display portion 12 depending on whether a caseof the fingertip portion being correctly placed or a case of thefingertip portion being not correctly placed.

The imaging device 10 shown in FIG. 2 includes two imaging positions 14a and 14 b, whereby, for example, fingerprints of one finger of bothhands can be simultaneously imaged. In the following, when it is notnecessary to distinguish between the imaging position 14 a and theimaging position 14 b, the imaging unit 1 a and the imaging unit 1 b,the display area 12 a and the display area 12 b, and the first imagingportion 11 a and the first imaging portion 11 b, they shall becollectively referred to by the reference symbols of the imagingposition 14, the imaging unit 1, the display area 12, and the firstimaging portion 11, respectively.

Next, a configuration example of the imaging unit 1 shown in FIG. 2 willbe described with reference to FIGS. 3 to 8B. FIG. 3 is a front viewschematically showing a detailed configuration example of the imagingunit 1 shown in FIG. 2. In FIG. 3, those configurations corresponding tothe configurations shown in FIG. 2 are denoted by the same referencesymbols. In the example shown in FIG. 3, the imaging position 14 isprovided on a contact surface 4 a of the optical element 4. A fingertipportion 7 is placed at the imaging position 14.

In the example shown in FIG. 3, the imaging unit 1 includes a firstlight source portion 2, a second light source portion 3, an opticalelement 4, and a first imaging portion 11. The first imaging portion 11includes a second imaging portion 6 and a third imaging portion 5.Further, a first image M1 that is an image captured by the first imagingportion 11 includes a second image M2 that is an image captured by thesecond imaging portion 6 and a third image M3 that is an image capturedby the third imaging portion 5.

The display portion 12 displays the first display image representing thefingertip portion 7, which is the object, based on the second image M2.The control portion 13 determines whether or not the fingertip portion7, which is the object, is correctly placed at the imaging position 14based on the third image M3.

In the example shown in FIG. 3, the optical element 4 is a prism. Theoptical element (prism) 4 has the contact surface 4 a, an intersectionsurface 4 b, an opposing surface 4 c that opposes the contact surface 4a, and a surface 4 d. The contact surface 4 a is the surface on whichthe fingertip portion 7, which is the object, comes into contact. Theintersection surface 4 b intersects the contact surface 4 a at apredetermined angle θ1. The opposing surface 4 c opposes the contactsurface 4 a. The surface 4 d opposes the intersecting surface 4 b. Inthis example, the optical element (prism) 4 is a polyhedron (forexample, a hexahedron) having a refractive index different from that ofair, and can be formed of, for example, glass, crystal, or the like. Inthe example shown in FIG. 3, the angle θ1 is an acute angle. Theopposing surface 4 c may or may not be parallel to the contact surface 4a. A black plate 41 is attached to the surface 4 d in order to enhancethe contrast of the image. However, the surface 4 d may be coated withblack paint or the like instead of the black plate 41. The opticalelement 4 is not limited to a prism. The optical element 4 may be alens, a reflector, an optical fiber, or a combination thereof capable ofguiding light rays from a light source to the object and guiding lightrays reflected by the object to the imaging unit 1.

In the arrangement of the second imaging portion 5 shown in FIG. 3, theangle 01 is an acute angle. However, the angle θ1 is not limited to anacute angle, and the angle θ1 may be an obtuse angle or a right angledepending on the arrangement of the imaging portions, for example, whenthe second imaging portion 5 is arranged below.

The first light source portion 2 is provided below the optical element 4and has a visible light LED 2 a such as a white LED (light emittingdiode). The first light source portion 2, that is, the visible light LED2 a, outputs a first light L1. In this case, the first light L1 containsa large amount of visible light components having a wavelength of about380 to 800 nm. Further, in the example shown in FIG. 3, the first lightL1 is irradiated from the opposing surface 4 c side of the opticalelement 4 (prism) to the fingertip portion 7, which is the object.

The second light source portion 3 is provided on the contact surface 4 aof the optical element 4 and has a plurality of infrared LEDs 3 a and 3b. The second light source portion 3, that is, the plurality of infraredLEDs 3 a and 3 b, outputs the second light L2 having a wavelength longerthan that of the first light L1. The second light L2 contains a largeamount of infrared components having a wavelength of about 800 nm to1000 nm. Further, in the example shown in FIG. 3, the second light L2 isirradiated from the peripheral portion on the contact surface 4 a to thefingertip portion 7, which is the object.

The second imaging portion 6 images the fingertip portion 7, which isthe object, from the opposing surface 4 c side. The second imagingportion 6 has an imaging element such as a CMOS (complementary metaloxide semiconductor) image sensor or a CCD (charge coupled device) imagesensor, and an infrared cut filter. The second imaging portion 6converts the visible light component of the input light into an imagesignal and outputs the image signal as the second image M2 to thecontrol portion 13. The second imaging portion 6 has low sensitivity tothe second light L2. That is, the second imaging portion 6 has apredetermined sensitivity to the first light L1 and does not have apredetermined sensitivity to the second light L2. However, the secondimaging portion 6 may not have an infrared cut filter. In this case, forexample, the same effect as when the infrared cut filter is provided maybe obtained by image processing or the like.

The third imaging portion 5 images the fingertip portion 7, which is theobject, from the intersection surface 4 b side. The third imagingportion 5 has an imaging element such as a CMOS image sensor or a CCDimage sensor. The third imaging portion 5 converts the visible lightcomponent and the infrared component of the input light into an imagesignal and outputs the image signal as the third image M3 to the controlportion 13. The third imaging portion 5 is highly sensitive to the firstlight L1 and the second light L2. That is, the third imaging portion 5has a predetermined sensitivity to the first light L1 and the secondlight L2.

FIG. 4 is a perspective view schematically showing an arrangementexample of each part of the imaging unit 1 shown in FIG. 3. In FIG. 4,those configurations corresponding to the configurations shown in FIG. 3are denoted by the same reference symbols. In the imaging unit 1 shownin FIG. 4, the first light source portion 2 is composed of four whiteLEDs 2 a, 2 b, 2 c and 2 d arranged on the opposing surface 4 c side ofthe optical element (prism) 4. The four white LEDs 2 a, 2 b, 2 c and 2 dare mounted on a substrate 43. The substrate 43 supports the opticalelement (prism) 4 from the opposing surface 4 c side to fix them.Further, the second light source portion 3 is composed of eight infraredLEDs 3 a, 3 b, 3 c, 3 d, 3 e, 3 f, 3 g and 3 h arranged in theperipheral portion on the contact surface 4 a of the optical element(prism) 4. The eight infrared LEDs 3 a, 3 b, 3 c, 3 d, 3 e, 3 f, 3 g and3 h are mounted on a substrate 42. The substrate 42 supports the opticalelement (prism) 4 from the contact surface 4 a side to fix them. Thenumber of LEDs is not limited to the example shown in FIG. 4.

Next, each optical path in the optical element (prism) 4 shown in FIG. 3will be described with reference to FIGS. 5 and 6. FIG. 5 is an enlargedschematic view showing the contact surface 4 a and the fingertip portion7 shown in FIG. 3. FIG. 6 is a schematic view showing an enlarged viewof the optical element (prism) 4 and the fingertip portion 7 shown inFIG. 3. As described above, one object of the imaging unit 1 of thesecond example embodiment is to capture the image of a fingerprint,which is one of the biological features. A fingerprint is composed of acombination of projecting and recess (valley line portions 72 and ridgeline portions 71) on the ventral surface 7 a of the fingertip portion 7.The first light L1 output by the first light source portion 2 isreflected by the ridge line portions 71 and the valley line portions 72placed on the contact surface 4 a to be incident on the optical element(prism) 4. Further, the second light L2 output by the second lightsource portion 3 passes through the fingertip portion 7, is emitted fromthe ridge line portions 71 and the valley line portions 72 placed on thecontact surface 4 a, and is incident on the optical element (prism) 4from the contact surface 4 a.

As shown in FIG. 5, the ridge line portion 71 of the fingertip portion 7is in contact with the contact surface 4 a. For this reason, therefractive index of the light incident from the ridge line portion 71 orreflected by the ridge line portion 71 is substantially the same as therefractive index of the optical element (prism) 4 (glass or the like).Therefore, the light emitted from the ridge line portion 71 or reflectedby the ridge line portion 71 can be regarded as the same as the lightreflected by the contact surface 4 a. That is, this light is radiatedsubstantially evenly in almost all directions in the optical element(prism) 4 as indicated by the arrows of the dashed lines, and can reachall the regions below the contact surface 4 a.

On the other hand, the valley line portion 72 is not in contact with thecontact surface 4 a, and so there is an air layer 8 between the valleyline portion 72 and the contact surface 4 a. Accordingly, the lightemitted from the valley line portion 72 or the reflected light at thevalley line portion 72 is incident on the contact surface 4 a via theair layer 8. Here, the refractive index of air is 1.0. The refractiveindex of glass is generally around 1.5, being in the range of 1.3 to2.0. The refractive index of water and skin is 1.3 to 1.4. As describedabove, since the refractive indices differ, the refraction phenomenon ofthe emitted light and the reflected light from the valley line portion72 is different from the emitted light and the reflected light of theridge line portion 71. Specifically, the emitted light and the reflectedlight from the valley line portion 72 are not radiated in alldirections. Further, since the refraction angle β is smaller than theincident angle α, the light emitted from the valley line portion 72 andthe light reflected by the valley line portion 72 are biased towardlight heading downward in the optical element (prism) 4 as shown by thearrows of the dashed lines.

As described with reference to FIG. 5, the light from the ridge lineportion 71 is radiated substantially evenly from the contact surface 4 ain all directions, and the light from the valley line portion 72 isradiated in a manner biased downward from the contact surface 4 a.Further, the light from the valley line portion 72 is radiated in amanner biased downward. Therefore, as shown in FIG. 6, the incidentangle γ of the light from the valley line portion 72 onto theintersection surface 4 b is larger than that of the light from the ridgeline portion 71. Therefore, as compared with the case of the light fromthe ridge line portion 71, the proportion of the incident angle γ of thelight from the valley line portion 72 exceeding the critical angle δbecomes larger. Therefore, as compared with the case of the light fromthe ridge line portion 71, the ratio of the light from the valley lineportion 72 being totally reflected by the intersecting surface 4 b islarger. Therefore, the light from the ridge line portion 71 reaches thethird imaging portion 5 more than the light from the valley line portion72. Here, the third imaging portion 5 images the fingertip portion 7from the intersection surface 4 b side to obtain the third image M3.Accordingly, in the third image M3, the ridge line portion 71 is brightand the valley line portion 72 is dark, and therefore, the third imageM3 is a high-contrast fingerprint image (hereinafter referred to as ahigh-contrast image). Further, when imaging the fingertip portion 7, thefirst light L1 is irradiated to the fingertip portion 7 from theopposing surface 4 c side, while the second light L2 is emitted to thefingertip portion 7 from the peripheral portion of the contact surface 4a. For this reason, the peripheral portion of the fingertip portion 7,which is difficult for the first light L1 to reach, can be brightened bythe second light L2. Therefore, the third imaging portion 5 can image afingerprint in a state where the entire fingerprint is appropriatelyilluminated. Accordingly, according to the second example embodiment, itis possible to increase the contrast across the entire fingerprintimage.

On the other hand, the light from the ridge line portion 71 and thelight from the valley line portion 72 reach the second imaging portion 6at a substantially equal ratio as compared with the ratio of reachingthe third imaging portion 5. Here, the second imaging portion 6 imagesthe fingertip portion 7 from the opposing surface 4 c side to obtain thesecond image M2. The second image M2 is an image similar to that seenwhen the fingertip portion 7 placed on the contact surface 4 a isdirectly viewed from the opposing surface 4 c. The second image M2 is animage in which the infrared component is suppressed. Accordingly, theimage captured by the second imaging portion 6 is a natural image of thefingertip portion 7 (hereinafter termed a natural image). However, asdescribed above, the infrared component need not be suppressed dependingon the use of the image captured by the second imaging portion 6.

The high-contrast image (third image M3) captured by the third imagingportion 5 is used, for example, for collating fingerprints. At thattime, it is desirable that the third imaging portion 5 capture afingerprint image of a wider area so that many minutiae are present inthe image. In the second example embodiment, since the third imagingportion 5 can capture a fingerprint image having high contrastthroughout, for example, the accuracy of fingerprint collation can beeasily improved. On the other hand, the natural image (second image M2)captured by the second imaging portion 6 is used, for example, for thepurpose of determining whether the finger is fake or genuine. Moreover,the natural image (second image M2) captured by the second imagingportion 6 is used, for example, for displaying the placement state ofthe fingertip portion 7 at the imaging position 14 on the displayportion 12 in a substantially real-time manner.

FIG. 7 is a plan view schematically showing an example of the case wherethe fingertip portion 7 is correctly placed at the imaging position 14on the imaging unit 1 shown in FIG. 4. FIG. 8A shows an example of thethird image M3 captured by the third imaging portion 5. FIG. 8B shows anexample of the second image M2 captured by the second imaging portion 6.

Next, an operation example of the imaging device 10 described withreference to FIGS. 1 and 2 will be described with reference to FIGS. 9and 10. FIG. 9 is a flowchart showing an operation example of theimaging device 10 shown in FIGS. 1 and 2. FIG. 10 is a schematic viewshowing a display example of the display portion 12 of the imagingdevice 10 shown in FIG. 2. Portion (a) of FIG. 10 shows the image(display state) displayed in a display area 120 when the fingertipportion 7 is not placed at the imaging position 14. Portion (b) of FIG.10 shows the image (display state) displayed in the display area 120when the fingertip portion 7 is placed at the imaging position 14 butthe manner of placement of the fingertip portion 7 is not correct.Portion (c) of FIG. 10 shows the image (display state) displayed in thedisplay area 120 when the fingertip portion 7 is correctly placed at theimaging position 14.

The process shown in FIG. 9 is repeatedly executed at predeterminedintervals. Further, as shown in FIG. 2, when the first imaging portion11 has the first imaging portion 11 a and the first imaging portion 11b, the control portion 13 executes the process shown in FIG. 9 based onthe captured image of the first imaging portion 11 a and the processshown in FIG. 9 based on the captured image of the first imaging portion11 b sequentially or in parallel.

When the process shown in FIG. 9 is started, the control portion 13acquires the first image M1 output by the first imaging portion 11 (StepS11). Next, the control portion 13 determines whether or not thefingertip portion 7, which is the object, is correctly placed at theimaging position 14 (Step S12). In Step S12, the control portion 13determines whether or not the fingertip portion 7 is correctly placed atthe imaging position 14 as follows, for example. That is, the controlportion 13 first extracts the contour line of the fingertip portion 7from the first image M1. Then, the control portion 13 determines thatthe fingertip portion 7 is correctly placed at the imaging position 14when the position and orientation of the contour line are within apredetermined range. On the other hand, the control portion 13determines that the fingertip portion 7 is not correctly placed at theimaging position 14 when the contour line of the fingertip portion 7cannot be extracted from the first image M1 or when the position ororientation of the extracted contour line is not within thepredetermined range. Alternatively, the control portion 13 may determinethat the fingertip portion 7 is correctly placed when the distributionstate of pixel values of the first image M1 matches a predeterminedpattern, and may determine that the fingertip portion 7 is not correctlyplaced when not so. Alternatively, the control portion 13 may determinethat the fingertip portion 7 is correctly placed when the average valueor the dispersion value of the pixel values of the first image M1 iswithin a predetermined range, and may determine that the fingertipportion 7 is not correctly placed when not so. The value that serves asa reference for determination may be obtained, for example,experimentally or by simulation. Further, the determination in Step S12may be executed every time in the process shown in FIG. 9, or may beexecuted every two or more times.

Upon having determined in Step S12 that the fingertip portion 7 iscorrectly placed (in the case of “YES” in Step S13), the control portion13 displays the first display image and the second display image in thedisplay mode for the case of the fingertip portion 7 being correctlyplaced (Step S14), and ends the process shown in FIG. 9. On the otherhand, upon having determined in Step S12 that the fingertip portion 7 isnot correctly placed (in the case of “NO” in Step S13), the controlportion 13 displays the first display image and the second display imagein the display mode for the case of the fingertip portion 7 not beingcorrectly placed (Step S15), returns to Step S11, and repeats theprocesses of steps S11 to S13.

In Step S14 and Step S15, the control portion 13, for example, changesthe display mode of the display portion 12 as follows. That is, if notplaced correctly, the control portion 13 displays the first displayimage 91 and the second display image 92 as shown in portion (a) orportion (b) of FIG. 10. The first display image 91 is an actual imageshowing the state of the imaging position 14 based on the first image M1(at least one of the second image M2 and the third image M3). In portion(a) of FIG. 10, the display area 120 does not include the image of thefingertip portion 7. In portion (b) of FIG. 10, the display area 120includes the image of the fingertip portion 7 when not placed correctly.The second display image 92 is an image displayed superimposed on thefirst display image 91, and includes a contour image (guidance) 921 anda mask image 922. The contour image 921 is a pattern showing theposition of the contour of the fingertip portion 7 when the fingertipportion 7 is correctly placed at the imaging position 14. In portion (a)and portion (b) of FIG. 10, the contour image 921 is, for example,displayed in red. The mask image 922 is a pattern that masks (hides) atleast a part of the fingerprint, which is a physical feature portion.The mask image 922 is displayed, for example, to prevent a person otherthan the user of the imaging device 10 from imaging the user'sfingerprint image displayed on the display portion 12. In portion (a)and portion (b) of FIG. 10, the mask image 922 is displayed, forexample, in red.

When the fingertip portion 7 is correctly placed, the control portion 13displays the first display image 91 and the second display image 92 asshown in portion (c) of FIG. 10. The second display image 92 shown inportion (c) of FIG. 10 includes the contour image 921 and the mask image922 having the same shapes as in portion (a) and portion (b) of FIG. 10but a different color. When the fingertip portion 7 is correctly placed,the contour image 921 and the mask image 922 are displayed in blue, forexample. By the change in the color of the second display image, theuser can recognize that the fingertip portion 7 is correctly placed atthe imaging position 14. Moreover, the second display image 92 shown inportion (c) of FIG. 10 includes an image (display) showing that thefingertip portion 7 is correctly placed at the imaging position 14,specifically, a character image 923 expressing the character string of“OK”. The character image 923 is displayed in blue, for example. Theuser can also recognize that the fingertip portion 7 is correctly placedat the imaging position 14 by the inclusion of the character stringindicating the correctness in the second display image.

As a guide for changing this display, when confirming the placementpositions of both fingers, if one of the fingers is placed correctly,the display of the corresponding finger may be changed individually.Further, a display to the effect of completion may be displayed onlywhen both fingers are correctly placed, that is, placement of thefingers has been normally completed. With such a display, when it isnecessary to authenticate the images of a plurality of fingers (parts ofa living body), it is possible to more clearly determine that thenecessary images have been captured without omission.

If the arrangement of the optical elements (the configuration of theoptical path through which the image is transmitted) causes the capturedimage to be upside down or a mirror image, image processing is performedthat flips the top and bottom of the captured image or inverts themirror image as necessary.

When the first imaging portion 11 includes the second imaging portion 6and the third imaging portion 5, and the first image M1 includes thesecond image M2 captured by the second imaging portion 6 and the thirdimage M3 captured by the third imaging portion 5, as shown in FIG. 3,the display portion 12 and the control portion 13 may operate asfollows. That is, in this case, the display portion 12 can display thefirst display image 91 representing the fingertip portion 7, which isthe object, based on the second image M2. Further, the control portion13 can determine whether or not the fingertip portion 7, which is theobject, is correctly placed at the imaging position 14 based on thethird image M3.

Next, with reference to FIG. 11, an operation example of the imagingdevice 10 shown in FIGS. 1 and 2 when the imaging unit 1 is configuredas shown in FIG. 3 will be described. FIG. 11 is a flowchart showing anoperation example of the imaging device 10 shown in FIGS. 1 and 2 whenthe imaging unit 1 is configured as shown in FIG. 3. In this case, thefirst imaging portion 11 includes the two types of imaging portions,that is, the second imaging portion 6 and the third imaging portion 5.

The process shown in FIG. 11 is repeatedly executed at a predeterminedperiod. When the first imaging portion 11 has the first imaging portion11 a and the first imaging portion 11 b, as shown in FIG. 2, the controlportion 13 executes the process shown in FIG. 11 based on the capturedimage and the process shown in FIG. 11 based on the captured image ofthe first imaging portion 11 b sequentially or in parallel whiledisplaying the captured image of the first imaging portion 11 a in realtime.

When the process shown in FIG. 11 is started, the control portion(determination portion) 13 acquires the second image M2 output by thesecond imaging portion 6 and the third image M3 output by the thirdimaging portion 5 (Step S21). Next, the control portion 13 calculates aquality value of the third image M3 based on the third image M3 (StepS22). However, the quality value calculation process in Step S22 may beexecuted every time in the process shown in FIG. 11 or may be executedevery two or more times.

The quality value of the image is an index value indicating the qualityof the fingerprint image for fingerprint collation. As the image qualityvalue, for example, the NFIQ (NIST Fingerprint Image Quality) value(NIST (National Institute of Standards and Technology) Fingerprint ImageQuality value (NFIQ NISTIR 7151—Fingerprint Image Quality, NISTInteragency Report, August 2004)), a quality value based on the averageconcentration or dispersion value described in Japanese UnexaminedPatent Application, First Publication No. 2002-222424, a quality valuebased on the certainty of minutiae described in Japanese UnexaminedPatent Application, First Publication No. 2007-226746, and the like maybe used. If the quality value of the image is equal to or greater than apredetermined value, the image has sufficient quality to be used forfingerprint collation. Therefore, when the quality value of the image isequal to or greater than a predetermined value, this means that thefingertip portion 7 was imaged in a state of being correctly placed atthe imaging position 14.

The image quality value is not limited to the above-mentioned qualityvalues that evaluate the quality of the image itself. The image qualityvalue may simply be a value indicating the degree to which the finger isplaced in the correct position with respect to the guide (the contourimage 921) (that is, a value that is an index of the magnitude ofmisalignment).

When the quality value calculated in Step S22 exceeds a predeterminedthreshold value (in the case of “YES” in Step S23), the control portion13 causes the display portion 12 to display the next image (Step S24).That is, as shown in portion (c) of FIG. 10, in Step S24, the controlportion 13 displays the first display image 91 based on the second imageM2 and displays in blue the contour image 921, the mask image 922 andthe character image 923 expressing the character string “OK” as thesecond display image with being superimposed on the first display image(Step S24). On the other hand, when the quality value calculated in StepS22 does not exceed a predetermined threshold value (in the case of “NO”in Step S23), the control portion 13 causes the display portion 12 todisplay the next image (Step S25). That is, as shown for example inportion (a) or portion (b) of FIG. 10, in Step S24 the control portion13 displays the first display image 91 based on the second image M2 anddisplays in red the contour image 921 and the mask image 922 as thesecond display image with being superimposed on the first display image91 (Step S25). The control portion 13 ends the process shown in FIG. 11after the process of Step S24 or Step S25.

In actual operation, even if the finger is placed correctly, therequired quality value may not be obtainable due to the state of thefinger (an abnormal surface condition due to the effect of wrinkles,grime, dryness, and the like) or grime in the optical system. In such acase, repetition of collation may not be concluded, resulting in a delayof the work. For this reason, after a predetermined time has elapsed, orin the case of the process of NG (in the case of “NO” in Step S23) beingrepeated a predetermined number of times, it is desirable to perform aprocess such as a display that the fingerprint cannot be acquired, theturning on of a warning lamp, or the like. Further, instead of or inaddition to the warning lamp, notification by display or audio that aclerk should be called may be made. It should be noted that thisnotification operation may also be performed based on the detection ofsome intention display (speaking or operation of the call button) by theuser regardless of the number of repetitions of NG.

Note that from the viewpoint of preventing leakage of personalinformation due to peeping at the display screen or the like, it isdesirable that the image displayed on the display portion 12 be dataobtaining by transforming an image actually imaged by the first to thirdimaging portions 11, 5 and 6 and used for collation, or a dummy imagethat simply shows the outline of the fingertip portion or the like.

As this display image, for example, an image obtained by the followingprocesses (1) to (4) can be used.

Process (1): The position of the finger is detected in real time basedon the second image M2.

Process (2): The display portion 12 displays a frame serving as a guideand a diagram showing the front side of the finger as seen from thefingernail side.

Process (3): The diagram showing the front side of the finger is movedaccording to the position detected in Process (1).

Process (4): If the finger fits in the frame, it is determined that thefingertip portion 7 is correctly placed.

That is, the fingernail side is displayed in an animation manner, notthe image of the fingerprint side of the finger. By doing so, theanimation displayed on the display portion and the direction in whichthe actual user's finger moves can be made the same, and so betteroperability can be obtained.

Further, the image on the fingernail side may be displayed on thedisplay on the user side to indicate the position, while the image onthe fingerprint side may be displayed on the display on the staff side.

As described above, according to the second example embodiment, the usercan easily place the fingertip portion 7, which is the object, at thecorrect imaging position 14.

The first and second example embodiments may be modified as in thefollowing modifications 1 to 5.

Modification 1

In the above example embodiment, a display (display portion 101) forallowing the user to recognize the position of the finger is provided.Further, another display may be provided so that a clerk confirming thefingerprint or the like can recognize the position of the finger.Alternatively, the image of the user's finger may be displayed byswitching the screen display of the existing display used by a clerk, oron a part of a split screen. Moreover, the image may be transmitted anddisplayed via a wireless or wired communication path.

By visually checking the image of the finger in this way, the clerk cancheck the placement state of the user's finger, and it can be used forthe clerk to prompt the user to move the position or monitor forfraudulent activity.

Modification 2

When the user places a finger in the vicinity of the guide, the displaymay be changed as follows, instead of the mode of display in the aboveexample embodiments. As specific examples of changing the way the guidedisplay changes depending on the state of the finger when the finger isplaced near the guide, the display may be changed as follows.

(Example a) Default color when no finger is placed within the imagingrange: blue

(Example b) When the finger can be confirmed on the display screen buthas not reached the normal position: white

(Example c) When placed normally according to the guide: red

(Example d) When the contour of the finger cannot be confirmed normally:orange (Example e) In case of system error: yellow

Note that the mode of change may not be the color, and a change in thedisplay mode such as a change in shape could be notified to the user.

Modification 3

The fingerprint image may be fed back to the display and displayed inreal time. In this case, from the viewpoint of preventing peeping by athird party, it is desirable to perform a predetermined process on thefingerprint image to display only a silhouette or contour of the finger.

Modification 4

The natural image of a finger acquired by a reading portion may befurther transmitted to an external terminal and used for advanceddetermining processing such as false finger judgment. In this case, asfor the external terminal, transmission may be performed to a terminalincluding at least one of a display or an analysis portion, with theexternal terminal being provided with, for example, the followingfunctions.

(Function a) When only a display is provided, a visual judgment can bemade by a clerk.

(Function b) If an analysis portion is provided, false fingerdetermination by some algorithm is also possible.

(Function c) When a display and an analysis portion are provided, inaddition to the above functions a and b, a function may be provided inwhich a clerk views the result of the analysis portion and makes a finaljudgment.

Modification 5

As the example embodiments of the present invention, the case of afingerprint image as an image including biological features has beendescribed, but the biological feature included in the captured image isnot limited thereto. For example, it may be a biological feature such asa vein, a skin pattern, or a palm print. In this case, the quality valuemay be a value indicating the degree to which the image including theabove-mentioned biological feature is suitable for collation, inaddition to the index value indicating the quality of the fingerprintimage.

Although the example embodiments of the present invention have beendescribed above with reference to the accompanying drawings, thespecific configuration is not limited to the above example embodiments,and includes design changes and the like within a range that does notdeviate from the gist of the present invention.

This application is based upon and claims the benefit of priority fromJapanese patent application No. 2018-218343, filed Nov. 21, 2018, thedisclosure of which is incorporated herein in its entirety by reference.

INDUSTRIAL APPLICABILITY

The present invention may be applied to an imaging device and an imagingmethod.

REFERENCE SYMBOLS

-   -   1: Imaging unit    -   2: First light source portion    -   2 a to 2 d: Visible light LED    -   3: Second light source portion    -   3 a to 3 h: Infrared LED    -   4: Optical element (prism)    -   4 a: Contact surface    -   4 b: Intersecting surface    -   4 c: Opposing surface    -   5: Third imaging portion    -   6: Second imaging portion    -   10: Imaging device    -   11, 11 a, 11 b: First imaging portion    -   12: Display portion    -   13: Control portion    -   14 14 a, 14 b: Imaging position    -   15: Determination portion    -   91: First display image    -   92: Second display image    -   921: Contour image    -   922: Mask image    -   923: Character image    -   L1: First light    -   L2: Second light

What is claimed is:
 1. An imaging device comprising: a camera thatcaptures an image of a portion of a living body to take in the image; adisplay that displays a first display image and a second display imagewith being superimposed on each other, the first display image beingbased on the image taken in by the camera, the second display imageincluding guidance regarding a way to place the portion of the livingbody in a prescribed position; at least one memory configured to storeinstructions; and at least one processor configured execute theinstructions to: determine whether the portion of the living body isplaced in the prescribed position; and until it is determined that theportion of the living body is placed in the prescribed position, causethe camera to newly capture and take in a new image of the portion ofthe living body and cause the display to display, as the first displayimage, an image based on the new taken in image.
 2. The imaging deviceaccording to claim 1, wherein determining whether the portion of theliving body is placed in the prescribed position comprises determiningthat the portion of the living body is placed at the prescribed positionwhen a quality value of the image captured by the camera is equal to orgreater than a predetermined threshold value.
 3. The imaging deviceaccording to claim 2, wherein the first display image is an image inwhich an infrared component is cut off.
 4. The imaging device accordingto claim 1 wherein the at least one processor is configured execute theinstructions to change a color of the second display image when adisplay mode in the display is changed.
 5. The imaging device accordingto claim 1, wherein the portion of the living body includes a physicalfeature portion for biometric authentication; and the second displayimage includes a mask image that masks at least a part of the physicalfeature portion.
 6. The imaging device according to claim 1, wherein theportion of the living body is a fingertip portion.
 7. The imaging deviceaccording to claim 1, wherein the at least one processor is configuredexecute the instructions to: cause the display to display the guidancein a first display mode when it is determined that the portion of theliving body is not placed at the prescribed position; and cause thedisplay to display the guidance in a second display mode different fromthe first display mode when it is determined that the portion of theliving body is placed at the prescribed position.
 8. An imaging methodcomprising: capturing an image of a portion of a living body to take inthe image; displaying a first display image and a second display imagewith being superimposed on each other, the first display image beingbased on the taken-in image, the second display image including guidanceregarding a way to place the portion of the living body in a prescribedposition; determining whether the portion of the living body is placedin the prescribed position; and until it is determined that that theportion of the living body is placed in the prescribed position, newlycapturing and taking in a new image of the portion of the living bodyand displaying, as the first display image, an image based on the newtaken-in image.